Skip to content

fix(performance): compile the example's tests again and gate them per PR - #900

Closed
FlorentinDUBOIS wants to merge 2 commits into
mainfrom
fix/performance-example-tests
Closed

FlorentinDUBOIS wants to merge 2 commits into
mainfrom
fix/performance-example-tests

Conversation

@FlorentinDUBOIS

@FlorentinDUBOIS FlorentinDUBOIS commented Nov 3, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

The scheduled performance workflow has failed every night since 2026-10-13. Two causes, one masking the other:

Nights Failing jobs Cause
2026-10-13 → 10-16 measurement workers (moonpool, workspace shards) unknown — those runs' artifacts have expired; prepare was green
2026-10-17 → 11-03 performance inputs and contracts (docker child exit 101) this PR

c4a5be6 (PIP-460 M1 stream consumer) removed MessageId::segment_id. The scalable-topics block in crates/magnetar/examples/performance.rs still assigned it, so the example's test module stopped compiling:

error[E0609]: no field `segment_id` on type `magnetar::MessageId`
   --> crates/magnetar/examples/performance.rs:618:23

Reproduced locally with the exact contracts command: cargo test -p magnetar-driver --example performance --release --features moonpool,scalable-topics --locked → exit 101.

Fix

  • Remove the dead block. It asserted that marker_position_matches separates ids differing only by segment; MessageId no longer carries a segment by design (see its doc comment), and the example drives no scalable workload. marker_position_matches is unchanged.
  • Close the gap that let c4a5be6 merge green: no per-PR job compiled an example's #[cfg(test)] module (clippy --all-targets checks examples as binaries; the workspace test run skips examples). The non-e2e test job now runs cargo test -p magnetar-driver --all-features --locked --examples, and CONTRIBUTING.md / CLAUDE.md carry the same line in their validation chains.

Verification

  • Exact contracts command, fixed tree → test result: ok. 4 passed; 0 failed.
  • New CI step on the pre-fix example → exit 101, same E0609 (the guard is seen red); on the fixed tree → 4 passed in ~100 s.
  • CONTRIBUTING.md $FEATURES variant → 4 passed.
  • cargo +nightly fmt --all --check, cargo clippy -p magnetar-driver --all-features --examples --profile test -- -D warnings, prettier, actionlint (only the pre-existing ubuntu-26.04 label notice, identical on main).

Second masked cause (found by the first dispatch, run 41624715389)

With the example compiling, prepare reached its Python contracts and failed one test (CI python-contracts.stderr: FAILED (failures=1)):

AssertionError: 'fixture-required' != 'fixture-free' : pulsar_fixture_contract.rs

c0d62a4 (landed in the same 0cfbd30 range as c4a5be6) added a_standalone_binary_port_is_refused_before_a_proxy_starts, which reached the proxy guard through Pulsar::start_proxy(). The Docker-free binary stayed container-free only because the assertion panicked first, and the fixture policy reads that .start_proxy() call as a fixture start. The guard now lives in Pulsar::proxy_command() (the proxy counterpart of command()), start_proxy() calls it before creating a container, and the test calls it as a plain #[test].

  • Classifier contract on the old test file → FAILED (failures=1) with the same assertion; on the new one → OK.
  • cargo test -p magnetar-driver --all-features --locked --test pulsar_fixture_contract → 22 passed; clippy --tests -D warnings clean.

Still to prove

A second manual performance dispatch (run 41626712212, head fbde666) is running to show whether the 2026-10-13 worker failure still exists. Note the first green nightly after merge will measure from a154636 (the last successful nightly's head), so a wide comparison range is expected, not a regression.

c4a5be6 (PIP-460 M1 stream consumer) removed `MessageId::segment_id`:
a scalable value now qualifies an ordinary id with an explicit segment
source instead of changing its equality. The `scalable-topics` block in
`examples/performance.rs` still assigned that field, so its test module
stopped compiling and the nightly `performance inputs and contracts`
step has exited 101 every night since 2026-10-17.

The block asserted that `marker_position_matches` separates two ids that
differ only by segment. That property no longer exists on `MessageId`,
and the example drives no scalable workload, so the block is removed;
`marker_position_matches` is unchanged.

No per-PR job compiled an example's `#[cfg(test)]` module: clippy's
`--all-targets` checks examples as binaries, and the workspace test run
skips examples. The non-e2e test job and the documented local chains
now run `cargo test -p magnetar-driver --examples`, which fails on the
pre-fix tree with the same E0609.

Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

c0d62a4 added `a_standalone_binary_port_is_refused_before_a_proxy_starts`
to `pulsar_fixture_contract.rs`, whose contract is Docker-free. It
reached the guard through `Pulsar::start_proxy()`, so the binary stayed
container-free only because the assertion panicked first: a regressed
guard would have launched a proxy instead of failing. The performance
fixture policy reads that `.start_proxy()` call as a fixture start and
classifies the target `fixture-required`, which fails the nightly's
`test_fixture_policy_classifies_every_target_of_the_repository_tests`
contract since 2026-10-17.

The guard moves into `Pulsar::proxy_command()`, the proxy counterpart
of `command()`; `start_proxy()` calls it before creating a container,
and the test calls it directly as a plain `#[test]`.

Signed-off-by: Florentin Dubois <florentin.dubois@clever.cloud>
@FlorentinDUBOIS

Copy link
Copy Markdown
Collaborator Author

Closing: the performance nightly fix is dropped for now. The worker shards still fail (exit code 2) for a separate cause dating from 2026-10-13, which this PR does not address.

@FlorentinDUBOIS
FlorentinDUBOIS deleted the fix/performance-example-tests branch November 3, 2026 15:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant